Skip to content

feat(replay): Add ReplaySnapshotObserver for snapshot testing#5386

Open
runningcode wants to merge 11 commits into
mainfrom
no/java-504-replay-before-store-frame-callback
Open

feat(replay): Add ReplaySnapshotObserver for snapshot testing#5386
runningcode wants to merge 11 commits into
mainfrom
no/java-504-replay-before-store-frame-callback

Conversation

@runningcode
Copy link
Copy Markdown
Contributor

@runningcode runningcode commented May 8, 2026

Summary

  • Add an experimental ReplaySnapshotObserver functional interface that fires right before a replay frame is stored to disk, receiving the masked bitmap, timestamp, and screen name
  • Expose snapshotObserver on ReplayIntegration so callers can set it after SDK init
  • Copy the bitmap before passing it to the observer so consumers own the copy and don't need to worry about the replay system's internal bitmap lifecycle
  • Add ReplaySnapshotTest UI integration test that captures masked replay frames on Sauce Labs
  • Enable useTestStorageService and *.png artifact collection in the Sauce Labs config
  • Move ReplaySnapshotTest to a conditional androidTestReplay source set so it compiles only when APPLY_SENTRY_INTEGRATIONS=true

Relates to JAVA-504

🤖 Generated with Claude Code

@linear-code
Copy link
Copy Markdown

linear-code Bot commented May 8, 2026

JAVA-504

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

Messages
📖 Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by 🚫 dangerJS against 4a829fe

@sentry
Copy link
Copy Markdown

sentry Bot commented May 8, 2026

📲 Install Builds

Android

🔗 App Name App ID Version Configuration
SDK Size io.sentry.tests.size 8.41.0 (1) release

⚙️ sentry-android Build Distribution Settings

@runningcode runningcode force-pushed the no/java-504-replay-before-store-frame-callback branch 2 times, most recently from 5b10cdd to e7452f7 Compare May 8, 2026 08:36
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 308.40 ms 356.90 ms 48.50 ms
Size 0 B 0 B 0 B

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
319f256 317.53 ms 370.83 ms 53.29 ms
ab8a72d 316.24 ms 356.38 ms 40.14 ms
cf708bd 434.73 ms 502.96 ms 68.22 ms
91bb874 310.68 ms 359.24 ms 48.56 ms
f634d01 375.06 ms 420.04 ms 44.98 ms
62b579c 299.75 ms 364.84 ms 65.09 ms
6b019b7 403.90 ms 546.09 ms 142.19 ms
d364ace 384.53 ms 453.51 ms 68.98 ms
5b66efd 308.67 ms 363.85 ms 55.18 ms
ee35ac3 346.83 ms 435.48 ms 88.65 ms

App size

Revision Plain With Sentry Diff
319f256 1.58 MiB 2.19 MiB 619.79 KiB
ab8a72d 1.58 MiB 2.12 MiB 551.55 KiB
cf708bd 1.58 MiB 2.11 MiB 539.71 KiB
91bb874 1.58 MiB 2.13 MiB 559.07 KiB
f634d01 1.58 MiB 2.10 MiB 533.40 KiB
62b579c 0 B 0 B 0 B
6b019b7 0 B 0 B 0 B
d364ace 1.58 MiB 2.11 MiB 539.75 KiB
5b66efd 1.58 MiB 2.13 MiB 559.07 KiB
ee35ac3 1.58 MiB 2.13 MiB 558.77 KiB

Previous results on branch: no/java-504-replay-before-store-frame-callback

Startup times

Revision Plain With Sentry Diff
a9ea107 311.04 ms 361.61 ms 50.57 ms
c906754 338.11 ms 408.86 ms 70.75 ms
cd28dc9 316.30 ms 354.64 ms 38.34 ms
1cddad0 342.73 ms 424.14 ms 81.41 ms
62bcea4 359.22 ms 426.90 ms 67.67 ms

App size

Revision Plain With Sentry Diff
a9ea107 0 B 0 B 0 B
c906754 0 B 0 B 0 B
cd28dc9 0 B 0 B 0 B
1cddad0 0 B 0 B 0 B
62bcea4 0 B 0 B 0 B

@runningcode runningcode force-pushed the no/java-504-replay-before-store-frame-callback branch from e7452f7 to 2aff4b0 Compare May 8, 2026 08:46
@runningcode runningcode marked this pull request as ready for review May 8, 2026 12:18
Comment thread sentry-android-replay/src/main/java/io/sentry/android/replay/ReplayIntegration.kt Outdated
@Before
fun setup() {
// GH Actions emulators don't support capturing screenshots for replay
@Suppress("KotlinConstantConditions")
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied this from ReplayTest but why are we even running these on emulators in gh actions?

Copy link
Copy Markdown
Member

@markushi markushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Let's discuss first if the API should be experimental for internal, but other than that no blockers.

Comment thread sentry/src/main/java/io/sentry/SentryReplayOptions.java Outdated
* intercepting frames for testing (e.g., screenshot comparison tests) or custom processing. The
* callback receives the frame after masking has been applied.
*
* <p>The frame bitmap is passed via a {@link Hint} using the key {@link
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

Comment thread sentry-android-replay/src/main/java/io/sentry/android/replay/ReplayIntegration.kt Outdated
Copy link
Copy Markdown
Member

@0xadam-brown 0xadam-brown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this @runningcode. A few comments.

Big picture, did we think about defining a new ReplayScreenshotObserver interface inside the replay module (akin to ScreenshotRecorderCallback) rather than routing through the (universally available) SentryReplayOptions?

An interface in the replay module would let us avoid the Hint indirection, and it'd avoid the tension of putting @ApiStatus.Internal on an *Options member.

Thoughts?

Comment thread sentry/src/main/java/io/sentry/SentryReplayOptions.java Outdated
Comment thread sentry/src/main/java/io/sentry/SentryReplayOptions.java Outdated
* TypeCheckHint#REPLAY_FRAME_BITMAP}. On Android, retrieve it with: {@code hint.getAs(
* TypeCheckHint.REPLAY_FRAME_BITMAP, Bitmap.class)}.
*
* <p>The callback runs on a background thread (replay executor). Do not recycle the bitmap — it
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: We could beef this up to protect against misuse if we think it'd help:

 * <p>**Bitmap lifecycle:** The bitmap is owned by the replay system. It should be deemed 
 * read-only and valid solely for the duration of the callback. Do not write to it or recycle it. Do
 * not synchronize on it, store a reference to it, or access it after this method returns – copy
 * the pixel data (e.g., compress to a file) within this method if you need it later.
 *
 * <p>The callback runs on a background thread (the replay executor).

...or whatever makes sense.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i've updated it to copy the bitmap. let me know if you think the new docs make sense along with that change to copy it

runningcode and others added 8 commits May 13, 2026 09:43
Add an experimental callback that fires right before a replay frame is
stored to disk. The callback receives the masked bitmap (via Hint),
timestamp, and current screen name. This enables snapshot testing of
replay masking without needing to decode stored video segments.

Includes a Kotlin extension for ergonomic usage:
  options.sessionReplay.beforeStoreFrame { bitmap, ts, screen -> ... }

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…(JAVA-504)

Add ReplaySnapshotTest that uses the beforeStoreFrame callback to
capture masked replay frames during a Compose UI test. Frames are
written to the Downloads/sauce_labs_custom_screenshots/ directory,
which is the standard path Sauce Labs collects screenshots from.

CI changes:
- Add *.png to Sauce Labs artifact match patterns
- Upload collected replay snapshots via sentry-cli build snapshots

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…VA-504)

The Kotlin extension `beforeStoreFrame` comes from `sentry-android-replay`
which may not resolve in the UI test module. Use the Java callback API
directly instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…VA-504)

GH Actions emulators don't support screenshot capture for replay,
so the ReplaySnapshotTest needs the same assumeThat guard used by
ReplayTest. Also adds a changelog entry for the beforeStoreFrame
callback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Markus Hintersteiner <markus.hintersteiner@sentry.io>
…r (JAVA-504)

Move the frame observer API from the core sentry module to
sentry-android-replay so it can use Bitmap directly instead of
the Hint indirection. The new ReplaySnapshotObserver fun interface
lives in the replay module and is set on ReplayIntegration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…in test (JAVA-504)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…AVA-504)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@runningcode runningcode force-pushed the no/java-504-replay-before-store-frame-callback branch from 375389b to f2c0c49 Compare May 13, 2026 07:43

buildFeatures { buildConfig = true }

configurations.all { resolutionStrategy.force(libs.jetbrains.annotations.get()) }
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to dig deeper in to why we need to add this to all the modules that add libs.jetbrains.annotations another day.

…VA-504)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…AVA-504)

Move ReplaySnapshotTest to a conditional androidTestReplay source set
so it's only compiled when APPLY_SENTRY_INTEGRATIONS is true. The test
imports replay classes that aren't on the classpath otherwise.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@runningcode runningcode changed the title feat(replay): Add beforeStoreFrame callback for snapshot testing feat(replay): Add ReplaySnapshotObserver for snapshot testing May 13, 2026
@runningcode runningcode requested a review from markushi May 13, 2026 11:07
…VA-504)

Consumers of the observer API receive a copy of the bitmap instead of
the replay system's shared instance. This eliminates race conditions
and crashes when consumers store or use the bitmap asynchronously.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
observer.onSnapshotCaptured(copy, frameTimeStamp, screen)
} catch (e: Throwable) {
options.logger.log(ERROR, "Error in ReplaySnapshotObserver", e)
copy.recycle()
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not have thought of calling copy.recycle() if we get an exception here. AI is smart.

val file = File(snapshotsDir, "${name}_$frameTimestamp.png")
file.outputStream().use { out -> bitmap.compress(Bitmap.CompressFormat.PNG, 100, out) }
}
bitmap.recycle()
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the bitmap copy, we now have to make sure that we call bitmap.recycle() as should consumers of our API.

@runningcode runningcode requested a review from 0xadam-brown May 13, 2026 11:29
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4a829fe. Configure here.

captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp ->
val observer = snapshotObserver
if (observer != null) {
val copy = bitmap.copy(bitmap.config!!, false)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unguarded NPE from bitmap.config!! skips frame storage

Medium Severity

The bitmap.config!! non-null assertion on line 316 is outside the try-catch block. Android's Bitmap.getConfig() can return null (e.g., for certain decoded image formats). If it does, a KotlinNullPointerException propagates out of the lambda, preventing addFrame on line 326 from ever executing. The ReplayExecutorService catches the exception so it won't crash, but the replay frame is silently lost. Setting a snapshotObserver can degrade replay quality even when the observer itself is well-behaved.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 4a829fe. Configure here.

}

@Test
fun captureComposeReplayFrameSnapshots() {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just FYI, this is a pretty contrived test but it just makes sure we can capture a replay using the new snapshot observer api

private val lifecycleLock = AutoClosableReentrantLock()
private val lifecycle = ReplayLifecycle()

@Volatile public var snapshotObserver: ReplaySnapshotObserver? = null
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this awkward API is the consequence of moving the API to the sentry-andorid-replay api

captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp ->
val observer = snapshotObserver
if (observer != null) {
val copy = bitmap.copy(bitmap.config!!, false)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we copy the bitmap now

val observer = snapshotObserver
if (observer != null) {
val copy = bitmap.copy(bitmap.config!!, false)
if (copy != null) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no point in calling the API if we have a null bitmap

Copy link
Copy Markdown
Member

@0xadam-brown 0xadam-brown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx for the updates! A few comments (two more impactful one's I've flagged with ❗) – let me know if you have any questions.

private val lifecycleLock = AutoClosableReentrantLock()
private val lifecycle = ReplayLifecycle()

@Volatile public var snapshotObserver: ReplaySnapshotObserver? = null
Copy link
Copy Markdown
Member

@0xadam-brown 0xadam-brown May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗- Ah, I thought we were going the SentryReplayOptions route to avoid allowing folks to mutate configs after Sentry.init() returns (+ for enhanced visibility and better ergonomics)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, then I misunderstood, this is the same API as in your branch which is what I thought we agreed to: a97de37#diff-cb8ccc45cef66aca485d205c9e99c17c114b6319ab9b7d52173b52ee72b047a8R126

We just need to make it public here instead of private in your branch because it needs to be usable from other packages i.e. other apps.

private val lifecycleLock = AutoClosableReentrantLock()
private val lifecycle = ReplayLifecycle()

@Volatile public var snapshotObserver: ReplaySnapshotObserver? = null
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Experimental

captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp ->
val observer = snapshotObserver
if (observer != null) {
val copy = bitmap.copy(bitmap.config!!, false)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have our try block below enclose bitmap.copy(), esp given the !! 👍

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the only types of errors that can happen when copying the bitmap are memory errors and we shouldn't try to catch those.
As for the !! I don't see how bitmap.config can be null if it is a valid bitmap. Do you know how it could be null?

}
}
addFrame(bitmap, frameTimeStamp, screen)
}
Copy link
Copy Markdown
Member

@0xadam-brown 0xadam-brown May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗- It's not in the diff, so apologies for missing it the first time round, but I see there's an onScreenshotRecorded(File, Long) method as well as the Bitmap version above. I imagine we need to call the observer in both....

* **Bitmap lifecycle:** The bitmap is a copy owned by the caller. You may store it or use it on
* another thread. Call [Bitmap.recycle] when you no longer need it to free native memory promptly.
*
* The callback runs on a background thread (the replay executor).
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth emphasizing that folks should keep processing quick or hand off to another thread. (Otherwise they'll slow down the SDK's replay pipeline.)

captureStrategy?.onScreenshotRecorded(bitmap) { frameTimeStamp ->
val observer = snapshotObserver
if (observer != null) {
val copy = bitmap.copy(bitmap.config!!, false)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Newbie question, but are we able to observe the performance hit of copying on our end?

Esp if we want to expand our use-cases beyond testing or debugging at some point, it'd be good to know whether we need to optimize.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memory-wise, each bitmap is roughly 900kb and we produce one frame per second.

Comment thread CHANGELOG.md
### Features

- Add support to configure reporting historical ANRs via `AndroidManifest.xml` using the `io.sentry.anr.report-historical` attribute ([#5387](https://github.com/getsentry/sentry-java/pull/5387))
- Session Replay: Add `ReplaySnapshotObserver` for observing captured replay frames ([#5386](https://github.com/getsentry/sentry-java/pull/5386))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth noting that this is for use in snapshot tests and for debugging purposes and isn't currently optimized for production use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants